GRADE guidance 37: rating imprecision in a body of evidence on test accuracy.

OBJECTIVES: To provide guidance on rating imprecision in a body of evidence assessing the accuracy of a single test. This guide will clarify when Grading of Recommendations Assessment, Development and Evaluation (GRADE) users should consider rating down the certainty of evidence by one or more levels for imprecision in test accuracy. STUDY DESIGN AND SETTING: A project group within the GRADE working group conducted iterative discussions and presentations at GRADE working group meetings to produce this guidance. RESULTS: Before rating the certainty of evidence, GRADE users should define the target of their certainty rating. GRADE recommends setting judgment thresholds defining what they consider a very accurate, accurate, inaccurate, and very inaccurate test. These thresholds should be set after considering consequences of testing and effects on people-important outcomes. GRADE's primary criterion for judging imprecision in test accuracy evidence is considering confidence intervals (i.e., CI approach) of absolute test accuracy results (true and false, positive, and negative results in a cohort of people). Based on the CI approach, when a CI appreciably crosses the predefined judgment threshold(s), one should consider rating down certainty of evidence by one or more levels, depending on the number of thresholds crossed. When the CI does not cross judgment threshold(s), GRADE suggests considering the sample size for an adequately powered test accuracy review (optimal or review information size [optimal information size (OIS)/review information size (RIS)]) in rating imprecision. If the combined sample size of the included studies in the review is smaller than the required OIS/RIS, one should consider rating down by one or more levels for imprecision. CONCLUSION: This paper extends previous GRADE guidance for rating imprecision in single test accuracy systematic reviews and guidelines, with a focus on the circumstances in which one should consider rating down one or more levels for imprecision.

The impact of cardiovascular diagnostics and treatments on fall risk in older adults: a scoping review and evidence map.

BACKGROUND: We aimed to summarize the published evidence on the fall risk reducing potential of cardiovascular diagnostics and treatments in older adults. METHODS: Design: scoping review and evidence map. DATA SOURCES: Medline and Embase. ELIGIBILITY CRITERIA: all available published evidence; Key search concepts: "older adults," "cardiovascular evaluation," "cardiovascular intervention," and "falls." Studies reporting on fall risk reducing effect of the diagnostic/treatment were included in the evidence map. Studies that investigated cardiovascular diagnostics or treatments within the context of falls, but without reporting a fall-related outcome, were included in the scoping review for qualitative synthesis. RESULTS: Two articles on cardiovascular diagnostics and eight articles on cardiovascular treatments were included in the evidence map. Six out of ten studies concerned pacemaker intervention of which one meta-analyses that included randomized controlled trials with contradictory results. A combined cardiovascular assessment/evaluation (one study) and pharmacotherapy in orthostatic hypotension (one study) showed fall reducing potential. The scoping review contained 40 articles on cardiovascular diagnostics and one on cardiovascular treatments. It provides an extensive overview of several diagnostics (e.g., orthostatic blood pressure measurements, heart rhythm assessment) useful in fall prevention. Also, diagnostics were identified, that could potentially provide added value in fall prevention (e.g., blood pressure variability and head turning). CONCLUSION: Although the majority of studies showed a reduction in falls after the intervention, the total amount of evidence regarding the effect of cardiovascular diagnostics/treatments on falls is small. Our findings can be used to optimize fall prevention strategies and develop an evidence-based fall prevention care pathway. Adhering to the World guidelines on fall prevention recommendations, it is crucial to undertake a standardized assessment of cardiovascular risk factors, followed by supplementary testing and corresponding interventions, as effective components of fall prevention strategies. In addition, accompanying diagnostics such as blood pressure variability and head turning can be of added value.

Benefit-Harm Analysis for Informed Decision Making on Participating in Colorectal Cancer Screening: A Modeling Study.

OBJECTIVES: To facilitate informed decision making on participating in colorectal cancer (CRC) screening, we assessed the benefit-harm balance of CRC screening for a wide range of subgroups over different time horizons. METHODS: The study combined incidence proportions of benefits and harms of (not) participating in CRC screening estimated by the Adenoma and Serrated pathway to CAncer microsimulation model, a preference eliciting survey, and benefit-harm balance modeling combining all outcomes to determine the net health benefit of CRC screening over 10, 20, and 30 years. Probability of net health benefit was estimated for 210 different subgroups based on age, sex, previous participation in CRC screening, and lifestyle. RESULTS: CRC screening was net beneficial in 183 of 210 subgroups over 30 years (median probability [MP] of 0.79, interquartile range [IQR] of 0.69-0.85) across subgroups. Net health benefit was greater for men (MP 0.82; IQR 0.69-0.89) than women (MP 0.76; IQR 0.67-0.83) and for those without history of participation in previous screenings (MP 0.84; IQR 0.80-0.89) compared with those with (MP 0.69; IQR 0.59-0.75). Net health benefit decreased with increasing age, from MP of 0.84 (IQR 0.80-0.86) at age 55 to 0.61 (IQR 0.56-0.71) at age 75. Shorter time horizons led to lower benefit, with MP of 0.70 (IQR 0.62-0.80) over 20 years and 0.54 (IQR 0.48-0.67) over 10 years. CONCLUSIONS: Our benefit-harm analysis provides information about net health benefit of screening participation, based on important characteristics and preferences of individuals, which could assist screening invitees in making informed decisions on screening participation.

Cross-Sectional Evaluation of Open Science Practices at Imaging Journals: A Meta-Research Study.

Objective: To evaluate open science policies of imaging journals, and compliance to these policies in published articles. Methods: From imaging journals listed we extracted open science policy details: protocol registration, reporting guidelines, funding, ethics and conflicts of interest (COI), data sharing, and open access publishing. The 10 most recently published studies from each journal were assessed to determine adherence to these policies. We calculated the proportion of open science policies into an Open Science Score (OSS) for all journals and articles. We evaluated relationships between OSS and journal/article level variables. Results: 82 journals/820 articles were included. The OSS of journals and articles was 58.3% and 31.8%, respectively. Of the journals, 65.9% had registration and 78.1% had reporting guideline policies. 79.3% of journals were members of COPE, 81.7% had plagiarism policies, 100% required disclosure of funding, and 97.6% required disclosure of COI and ethics approval. 81.7% had data sharing policies and 15.9% were fully open access. 7.8% of articles had a registered protocol, 8.4% followed a reporting guideline, 77.4% disclosed funding, 88.7% disclosed COI, and 85.6% reported ethics approval. 12.3% of articles shared their data. 51% of articles were available through open access or as a preprint. OSS was higher for journal with DOAJ membership (80% vs 54.2%; P < .0001). Impact factor was not correlated with journal OSS. Knowledge synthesis articles has a higher OSS scores (44.5%) than prospective/retrospective studies (32.6%, 30.0%, P < .0001). Conclusion: Imaging journals endorsed just over half of open science practices considered; however, the application of these practices at the article level was lower.

Diagnostic accuracy of qPCR and microscopy for cutaneous leishmaniasis in rural Ecuador: A Bayesian latent class analysis.

BACKGROUND: Clinical and laboratory diagnosis of cutaneous leishmaniasis (CL) is hampered by under-ascertainment of direct microscopy. METHODS: This study compared the diagnostic accuracy of qPCR on DNA extracted from filter paper to the accuracy of direct smear slide microscopy in participants presenting with a cutaneous lesion suspected of leishmaniasis to 16 rural healthcare centers in the Ecuadorian Amazon and Pacific regions, from January 2019 to June 2021. We used Bayesian latent class analysis to estimate test sensitivity, specificity, likelihood ratios (LR), and predictive values (PV) with their 95% credible intervals (95%CrI). The impact of sociodemographic and clinical characteristics on predictive values was assessed as a secondary objective. RESULTS: Of 320 initially included participants, paired valid test results were available and included in the diagnostic accuracy analysis for 129 from the Amazon and 185 from the Pacific region. We estimated sensitivity of 68% (95%CrI 49% to 82%) and 73% (95%CrI 73% to 83%) for qPCR, and 51% (95%CrI 36% to 66%) and 76% (95%CrI 65% to 86%) for microscopy in the Amazon and Pacific region, respectively. In the Amazon, with an estimated disease prevalence among participants of 73%, negative PV for qPCR was 54% (95%CrI 5% to 77%) and 44% (95%CrI 4% to 65%) for microscopy. In the Pacific, (prevalence 88%) the negative PV was 34% (95%CrI 3% to 58%) and 37% (95%CrI 3% to 63%). The addition of qPCR parallel to microscopy in the Amazon increases the observed prevalence from 38% to 64% (+26 (95%CrI 19 to 34) percentage points). CONCLUSION: The accuracy of either qPCR on DNA extracted from filter paper or microscopy for CL diagnosis as a stand-alone test seems to be unsatisfactory and region-dependent. We recommend further studies to confirm the clinically relevant increment found in the diagnostic yield due to the addition of qPCR.

Screening for Latent Tuberculosis Infection in Solid Organ Transplant Recipients to Predict Active Disease: A Systematic Review and Meta-Analysis of Diagnostic Studies.

Background: This is a systematic review and meta-analysis of diagnostic test accuracy studies to assess the predictive value of both tuberculin skin test (TST) and interferon-gamma release assays (IGRA) for active tuberculosis (TB) among solid organ transplantation (SOT) recipients. Methods: Medline, Embase, and the CENTRAL databases were searched from 1946 until June 30, 2022. Two independent assessors extracted data from studies. Sensitivity analyses were performed to investigate the effect of studies with high or low risk of bias. Methodological quality of each publication was assessed using QUADAS-2. Results: A total of 43 studies (36 403 patients) with patients who were screened for latent TB infection (LTBI) and who underwent SOT were included: 18 were comparative and 25 noncomparative (19 TST, 6 QuantiFERON-TB Gold In-Tube [QFT-GIT]). For IGRA tests taken together, positive predictive value (PPV) and negative predictive value (NPV) were 1.2% and 99.6%, respectively. For TST, PPV was 2.13% and NPV was 95.5%. Overall, PPV is higher when TB burden is higher, regardless of test type, although still low in absolute terms. Incidence of active TB was similar between studies using LTBI prophylaxis (mean incidence 1.22%; 95% confidence interval [CI], .2179-2.221) and those not using prophylaxis (mean incidence 1.045%; 95% CI, 0.2731-1.817; P = .7717). Strengths of this study include the large number of studies available from multiple different countries; limitations include absence of gold standard for diagnosis of latent TB and low incidence of active TB. Conclusions: We found both TST and IGRA had a low PPV and high NPV for the development of active TB posttransplant. Further studies are needed to better understand how to prevent active TB in the SOT population.

Clinical Prediction Models and Predictors for Death or Adverse Neurodevelopmental Outcome in Term Newborns with Hypoxic-Ischemic Encephalopathy: A Systematic Review of the Literature.

BACKGROUND: Although many predictive parameters have been studied, an internationally accepted, validated predictive model to predict the clinical outcome of asphyxiated infants suffering from hypoxic-ischemic encephalopathy is currently lacking. The aim of this study was to identify, appraise and summarize available clinical prediction models, and provide an overview of all investigated predictors for the outcome death or neurodevelopmental impairment in this population. METHODS: A systematic literature search was performed in Medline and Embase. Two reviewers independently included eligible studies and extracted data. The quality was assessed using PROBAST for prediction model studies and QUIPS assessment tools for predictor studies. RESULTS: A total of nine prediction models were included. These models were very heterogeneous in number of predictors assessed, methods of model derivation, and primary outcomes. All studies had a high risk of bias following the PROBAST assessment and low applicability due to complex model presentation. A total of 104 predictor studies were included investigating various predictors, showing tremendous heterogeneity in investigated predictors, timing of predictors, primary outcomes, results, and methodological quality according to QUIPS. Selected high-quality studies with accurate discriminating performance provide clinicians and researchers an evidence map of predictors for prognostication after HIE in newborns. CONCLUSION: Given the low methodological quality of the currently published clinical prediction models, implementation into clinical practice is not yet possible. Therefore, there is an urgent need to develop a prediction model which complies with the PROBAST guideline. An overview of potential predictors to include in a prediction model is presented.

Over-interpretation of findings in diagnostic accuracy studies of infectious diseases.

OBJECTIVES: To assess the prevalence of overly positive interpretation, also called 'spin',-of results in diagnostic accuracy studies of infectious diseases and to identify suggestions for improvement. METHODS: A PubMed search was performed to identify diagnostic accuracy studies of infectious diseases published between January and March 2019. Each article was assessed by two authors independently to identify study characteristics and forms of actual and potential over-interpretation. 'Actual over-interpretation' was defined as conclusions that were not on the basis of study aims or conclusions that were more favourable than justified by the study findings. There are other practices that may result in the over-interpretation of study findings and these have been described as 'potential over-interpretation'. RESULTS: The final analysis included 120 studies. Favourable or promising recommendations were made in the main text of 101 (84%) of the included studies. Evidence of actual over-interpretation (spin) was found in 30 (25%) articles, with 22 (18%) studies reporting a conclusion that did not match the study aims and 56 (47%) studies with a more positive conclusion in the abstract than the main text. All analysed studies exhibited at least one form of potential over-interpretation, with was most commonly a lack of sample size calculation (n = 109, 91%) and not reporting a null hypothesis (n = 115, 96%). DISCUSSION: Evidence of over-interpretation of results was found in one-third of the included studies. We have proposed possible interventions to prevent overly positive interpretations of results in diagnostic accuracy studies.

Perceived Importance of the Benefits and Harms of Colorectal Cancer Screening: A Best-Worst Scaling Study.

OBJECTIVES: To elicit the relative importance of the benefits and harms of colorectal cancer (CRC) screening among potential screening participants in the Dutch population. METHODS: In a consensus meeting with 11 experts, risk reduction of CRC and CRC deaths (benefits) and complications from colonoscopy, stress of receiving positive fecal immunological test (FIT) results, as well as false-positive and false-negative FIT results (harms) were selected as determinant end points to consider during decision making. We conducted an online best-worst scaling survey among adults aged 55 to 75 years from the Dutch Health Care Consumer Panel of The Netherlands Institute for Health Services Research to elicit preference values for these outcomes. The preference values were estimated using conditional logit regression. RESULTS: Of 265 participants, 234 (89%) had ever participated in CRC screening. Compared with the stress of receiving a positive FIT result, the outcome perceived most important was the risk of CRC death (odds ratio [OR] 4.5; 95% confidence interval [CI] 3.9-5.1), followed by risk of CRC (OR 4.1; 95% CI 3.6-4.7), a false-negative FIT result (OR 3.1; 95% CI 2.7-3.5), colonoscopy complications (OR 1.6; 95% CI 1.4-1.8), and a false-positive FIT result (OR 1.4; 95% CI 1.3-1.6). The magnitude of these differences in perceived importance varied according to age, educational level, ethnic background, and whether the individual had previously participated in CRC screening. CONCLUSION: Dutch men and women eligible for FIT-based CRC screening perceive the benefits of screening to be more important than the harms.

Diagnostic accuracy and added value of blood-based protein biomarkers for pancreatic cancer: A meta-analysis of aggregate and individual participant data.

Background: Novel blood-based protein biomarkers may be of value for efficient, accurate, and non-invasive diagnosis of pancreatic cancer. This study assesses the diagnostic accuracy of newly recognized blood-based protein biomarkers for detecting pancreatic cancer, and investigates their added value to CA19-9, the common blood-based biomarker in clinical use for pancreatic cancer. Methods: PubMed, Embase, Web of Science, and the Wiley/Cochrane Library were systematically searched from inception until June 2022. A meta-analysis of aggregate and individual participant data was conducted using frequentist and Bayesian hierarchical random-effects models. The added clinical utility of protein biomarkers was investigated using bootstrap bias-corrected decision curve analyses. Findings: Aggregate data from 28 primary studies (6127 participants) were included, of which 8 studies (1790 participants) provided individual participant data. CA19-9 was significantly more accurate than MIC-1 for distinguishing pancreatic cancer from benign disease (AUC, 0.83 vs 0.74; relative diagnostic odds ratio [rDOR], 2.10 [95% CI, 0.98-4.48]; p = 0.002), THBS2 (AUC, 0.87 vs 0.69; rDOR, 4.53 [2.16-9.39]; p < 0.0001), TIMP-1 (AUC, 0.91 vs 0.70; rDOR, 8.00 [3.81-16.9]; p < 0.0001), OPN (AUC, 0.89 vs 0.74; rDOR, 4.22 [1.13-15.6]; p < 0.0001), ICAM-1 (AUC, 0.91 vs 0.68; rDOR 9.30 [0.87-99.5]; p < 0.0001), and IGFBP2 (AUC, 0.91 vs 0.68; rDOR, 4.48 [0.78-24.3]; p < 0.0001). The addition of these novel protein biomarkers to CA19-9 did not significantly improve the AUC, and resulted in minor increases or limited decreases in clinical utility. Interpretation: Novel protein biomarkers have moderate diagnostic accuracy, do not outperform CA19-9 in differentiating pancreatic cancer from benign disease, and show limited added clinical value to CA19-9. We propose recommendations to aid the development of minimally invasive diagnostic tests with sufficient clinical utility to improve the management of patients with suspected pancreatic cancer. Funding: Bennink Foundation, Dutch Cancer Foundation (KWF Kankerbestrijding), and AIRC.

Clinical criteria for Mucosal Leishmaniasis diagnosis in rural South America: A systematic literature review.

BACKGROUND: Mucosal Leishmaniasis (ML), a neglected tropical disease caused by Leishmania parasites, impairs the quality of life of under-resourced populations in South America. If not treated promptly, this disease progresses to facial deformities and death. The low sensitivity of microscopy results and the unavailability of other accurate tests hamper the diagnosis. As clinical criteria are readily available in any setting, these may be combined in a syndromic algorithm, which in turn can be used as a diagnostic tool. We explore potential clinical criteria for a syndromic diagnostic algorithm for ML in rural healthcare settings in South America. METHODOLOGY/PRINCIPAL FINDINGS: The protocol for this systematic review was pre-registered in PROSPERO with the number: CRD42017074148. In patients with ML, described in case series identified through a systematic retrieval process, we explored the cumulative ML detection rates of clinical criteria. Participants: all patients with active mucosal disease from an endemic area in South America. Any original, non-treatment study was eligible, and case reports were excluded. PUBMED, EMBASE, Web of Science, SCIELO, and LILACS databases were searched without restrictions. The risk of bias was assessed with the JBI checklist for case series. We included 10 full texts describing 192 ML patients. Male gender had the highest detection rate (88%), followed by ulcer of the nasal mucosa (77%), age >15 (69%), and symptom duration >4 months (63%). SIGNIFICANCE: Within this selection of patients, we found that the male gender, ulcer of the nasal mucosa, age >15, and symptom duration >4 months lead to the highest detection rates. However, higher detection comes -naturally- with a higher rate of false positives as well. As we only included ML patients, this could not be verified. Therefore, the criteria that we found to be most promising should be validated in a well-designed prospective study.

Impact of cardiovascular evaluations and interventions on fall risk in older adults: a protocol for a scoping review and evidence map.

INTRODUCTION: Cardiovascular disorders are increasingly recognised as important fall risk factors in older adults. Falls are a major public health problem in older adults, and therefore, effective interventions for reducing falls are essential for this population. Cardiovascular disease is a clinically relevant (but often overlooked) and potentially modifiable risk factor for falls. Literature describing the effects of cardiovascular assessments and treatments on fall prevention has generally focused on only one specific test or treatment. A comprehensive, comparative overview surrounding the effectiveness of available assessments and treatments on cardiovascular related fall risk is currently lacking. METHODS AND ANALYSIS: A scoping review and evidence map will be conducted to summarise the available evidence regarding the (comparative) effectiveness of cardiovascular assessments and therapeutic interventions on reducing fall risk in older individuals. A systematic and comprehensive literature search will be performed in MEDLINE and Embase using the key components 'older adults', 'cardiovascular evaluation', 'cardiovascular intervention' and 'falls'. Furthermore, we will create an evidence map to summarise the quantity and quality of currently available evidence identified in the scoping review. The evidence map will consider, but will not be limited to, observational studies, randomised controlled trials and reviews evaluating cardiovascular tests and treatments (vs controls) on fall risk in older adults. ETHICS AND DISSEMINATION: The scoping review and evidence map will only include data that are publicly available and, therefore, ethical approval is not required. The results will be submitted for publication in a peer-reviewed journal and presented at scientific conferences.

Outcome Prediction and Inter-Rater Comparison of Four Brain Magnetic Resonance Imaging Scoring Systems of Infants with Perinatal Asphyxia and Therapeutic Hypothermia.

INTRODUCTION: The brain magnetic resonance imaging (MRI) result is a major predictor for the outcome of term infants with perinatal asphyxia who underwent therapeutic hypothermia. In daily practice, no uniform method is used to assess these images. PURPOSE: The aim of this study was to determine which MRI-score best predicts adverse outcome at 24 months of age and has the highest inter-rater reliability. METHODS: Four MRI scoring systems for term infants with perinatal asphyxia were selected: Rutherford score, Trivedi score, Weeke score, and NICHD NRN score. Experienced blinded raters retrospectively evaluated the brain MR Images of 161 infants using all four scoring systems. Long-term outcome (the composite outcome death or adverse outcome, and its separate components) were routinely assessed by standardized testing at the age of 24 months. The predictive accuracy was assessed by logistic regression analyses and expressed as area under the ROC curve (AUC). The inter-rater reliability of the scores was calculated by the weighted Kappa or intraclass correlation. A sensitivity analysis using only high-quality MRI scans was performed. RESULTS: All four MRI scoring systems demonstrated an AUC of >0.66 for the prediction of adverse outcome and ≥0.80 for the prediction of death. The inter-rater reliability analyses demonstrated the highest reliability for the Weeke and Trivedi scores. When only assessing the high-quality scans, the AUC increased further. CONCLUSION: All four MRI brain scores proved reliable predictors for an adverse outcome at 24 months of age. The Weeke and Trivedi score demonstrated the highest inter-rater reliability. The use of high-quality MRI further improved prediction.